feat: trigger content claims from piece cid #230

vasco-santos · 2023-09-18T08:27:58Z

Needs:

Upgrade dep + Tests
Set env variables
feat: trigger aggregator from pieces computed #229

vasco-santos · 2023-09-19T09:27:14Z

test/integration.test.js

@@ -114,7 +114,7 @@ test('authorizations can be blocked by email or domain', async t => {
 })

 // Integration test for all flow from uploading a file to Kinesis events consumers and replicator
-test('w3infra integration flow', async t => {
+test.skip('w3infra integration flow', async t => {


todo once dep available

vasco-santos · 2023-09-19T09:28:09Z

filecoin/package.json

    "@ucanto/principal": "^8.1.0",
    "@ucanto/transport": "^8.0.0",
+    "@web3-storage/content-claims": "^3.0.1",


Needs storacha/content-claims#19

seed-deploy · 2023-09-19T09:38:11Z

View stack outputs

pr230-w3infra-CarparkStack

Name Value

BucketName carpark-pr230-0

Region us-east-2
pr230-w3infra-RoundaboutStack

Name Value

ApiEndpoint https://pa1kir5us7.execute-api.us-east-2.amazonaws.com

CustomDomain https://pr230.roundabout.web3.storage
pr230-w3infra-SatnavStack

Name Value

BucketName satnav-pr230-0

Region us-east-2
pr230-w3infra-UploadApiStack

Name Value

ApiEndpoint https://xz6cpbet41.execute-api.us-east-2.amazonaws.com

CustomDomain https://pr230.up.web3.storage
pr230-w3infra-BusStack
pr230-w3infra-FilecoinStack
pr230-w3infra-ReplicatorStack
pr230-w3infra-UcanInvocationStack
pr230-w3infra-UploadDbStack

alanshaw · 2023-09-19T10:15:23Z

filecoin/functions/piece-cid-report.js

+    did: aggregatorDid,
+    url: aggregatorUrl


Suggested change

did: aggregatorDid,

url: aggregatorUrl

did: contentClaimsDid,

url: contentClaimsUrl

alanshaw · 2023-09-19T10:17:21Z

filecoin/functions/piece-cid-report.js

+    claimsInvocationConfig: /** @type {import('../types').ClaimsInvocationConfig} */ ({
+      issuer,
+      audience: claimsServiceConnection.id,
+      with: issuer.did(),


You either need to use the claims service private key or delegate assert/equals to this issuer from the claims service (and include it as a proof).

Added private key as secret as we talked

test/integration.test.js

alanshaw

Code changes LGTM 🚀

README.md

alanshaw · 2023-09-20T08:47:44Z

README.md

+Generated by [@ucanto/principal `EdSigner`](https://github.com/web3-storage/ucanto) via [`ucan-key`](https://www.npmjs.com/package/ucan-key)
+
+_Example:_ `MgCZG7EvaA...1pX9as=`
+


One thing I did when integrating claims with the current web3.storage API was allow a proof to be passed.

You need to sign a UCAN with a private key and provide proof that you have the capability. When the private key is the private key of the content claims service no proof is needed (you are self signing). When it's a different private key you need to have a delagation for assert/equals as proof.

By allowing a proof to be passed you enable both cases. It just means we can switch to the latter case in the future without code changes.

Not blocking...but should be easy to implement:

https://github.com/web3-storage/web3.storage/blob/1cc9707af6a10bccec4dd93888cf1d5fc1a8c0cb/packages/api/src/env.js#L232-L248

Co-authored-by: Alan Shaw <[email protected]>

vasco-santos · 2023-09-20T10:14:56Z

Written content claims to staging 🎉

➜  content-claims git:(main) ✗ node packages/cli/bin.js read bagbaiera7xsk333k7wm67rbgcq7lxncqhbil2w725m6clcb7rzg7wqkolxwa --verbose 
WARN: using did:web:staging.claims.web3.storage
�㖲�p\m���}ߵroots��*X%q <6bX�����T!�>�Hx���y�9�]�@gversion�q  �X¾�g6��a�`�;7����asXD��@:Z�-q;Pz(        �k�p�+  L�,��]�i�|U�
�6=gcontent�*X&� ���j����&>��P8P�[��<%�?�M�AN]�ccanmassert/equalsdwithx#did:web:staging.claims.web3.storagecaudX!�web:staging.claims.web3.storagecexpe
��cissX!�web:staging.claims.web3.storagecprf�Yq <6bX�����T!�>�Hx���y�9�]�@�[email protected]�*X%q  �X¾�g6��a�`�;7���{
  iss: 'did:web:staging.claims.web3.storage',
  aud: 'did:web:staging.claims.web3.storage',
  v: '0.9.1',
  s: {
    '/': {
      bytes: '7aEDQDpaiy1xO1B6KAmxa9xwqysJTN8s0OVd0mkcoXxV8w0d/OOWsqVwXG38gY1937UMLvRJjbMvK5WIBM1UN3pHjwQ'
    }
  },
  exp: 1695203468,
  att: [
    {
      can: 'assert/equals',
      nb: {
        content: {
          '/': 'bagbaiera7xsk333k7wm67rbgcq7lxncqhbil2w725m6clcb7rzg7wqkolxwa'
        },
        equals: {
          '/': 'bafkzcibbandcag7o7h6n5elb4o5felutjoss53wyhi7otjvhz4mpmaqn4a3d2'
        }
      },
      with: 'did:web:staging.claims.web3.storage'
    }
  ],
  prf: [],
  '/': 'bafyreia32c3cbzkyyk7jezzwawoncyo2malkagomxibb2chnhm3qjmmuxe'
}
➜  content-claims git:(main) ✗ 
➜  content-claims git:(main) ✗ 
➜  content-claims git:(main) ✗ node packages/cli/bin.js read bafkzcibbandcag7o7h6n5elb4o5felutjoss53wyhi7otjvhz4mpmaqn4a3d2 --verbose
WARN: using did:web:staging.claims.web3.storage
�㖲�p\m���}ߵroots��*X%q <6bX�����T!�>�Hx���y�9�]�@gversion�q  �X¾�g6��a�`�;7����asXD��@:Z�-q;Pz(        �k�p�+  L�,��]�i�|U�
�6=gcontent�*X&� ���j����&>��P8P�[��<%�?�M�AN]�ccanmassert/equalsdwithx#did:web:staging.claims.web3.storagecaudX!�web:staging.claims.web3.storagecexpe
��cissX!�web:staging.claims.web3.storagecprf�Yq <6bX�����T!�>�Hx���y�9�]�@�[email protected]�*X%q  �X¾�g6��a�`�;7���{
  iss: 'did:web:staging.claims.web3.storage',
  aud: 'did:web:staging.claims.web3.storage',
  v: '0.9.1',
  s: {
    '/': {
      bytes: '7aEDQDpaiy1xO1B6KAmxa9xwqysJTN8s0OVd0mkcoXxV8w0d/OOWsqVwXG38gY1937UMLvRJjbMvK5WIBM1UN3pHjwQ'
    }
  },
  exp: 1695203468,
  att: [
    {
      can: 'assert/equals',
      nb: {
        content: {
          '/': 'bagbaiera7xsk333k7wm67rbgcq7lxncqhbil2w725m6clcb7rzg7wqkolxwa'
        },
        equals: {
          '/': 'bafkzcibbandcag7o7h6n5elb4o5felutjoss53wyhi7otjvhz4mpmaqn4a3d2'
        }
      },
      with: 'did:web:staging.claims.web3.storage'
    }
  ],
  prf: [],
  '/': 'bafyreia32c3cbzkyyk7jezzwawoncyo2malkagomxibb2chnhm3qjmmuxe'
}

seed-deploy bot had a problem deploying to pr230 September 18, 2023 08:28 Failure

vasco-santos force-pushed the feat/trigger-content-claims-from-piece-cid branch from 6c34000 to 3df02f1 Compare September 18, 2023 16:57

seed-deploy bot had a problem deploying to pr230 September 18, 2023 16:57 Failure

vasco-santos force-pushed the feat/trigger-content-claims-from-piece-cid branch from 3df02f1 to 607fa87 Compare September 19, 2023 09:27

seed-deploy bot temporarily deployed to pr230 September 19, 2023 09:27 Inactive

vasco-santos commented Sep 19, 2023

View reviewed changes

vasco-santos force-pushed the feat/trigger-content-claims-from-piece-cid branch from 607fa87 to 0ad12d8 Compare September 19, 2023 09:28

seed-deploy bot temporarily deployed to pr230 September 19, 2023 09:38 Inactive

alanshaw requested changes Sep 19, 2023

View reviewed changes

seed-deploy bot had a problem deploying to pr230 September 19, 2023 13:31 Failure

seed-deploy bot had a problem deploying to pr230 September 19, 2023 13:40 Failure

seed-deploy bot had a problem deploying to pr230 September 19, 2023 13:45 Failure

seed-deploy bot temporarily deployed to pr230 September 19, 2023 13:54 Inactive

vasco-santos requested a review from alanshaw September 19, 2023 13:59

vasco-santos marked this pull request as ready for review September 19, 2023 14:00

vasco-santos commented Sep 19, 2023

View reviewed changes

test/integration.test.js Outdated Show resolved Hide resolved

seed-deploy bot temporarily deployed to pr230 September 19, 2023 14:04 Inactive

alanshaw approved these changes Sep 20, 2023

View reviewed changes

seed-deploy bot temporarily deployed to pr230 September 20, 2023 09:11 Inactive

seed-deploy bot temporarily deployed to pr230 September 20, 2023 09:54 Inactive

vasco-santos and others added 7 commits September 20, 2023 11:58

feat: trigger content claims equals on piece cid computed

74c2de6

chore: address review comments

73223f3

chore: use final version of content-claims client

3af33b1

chore: satisfy types for generated secrets

7625e2c

chore: run full interop tests suite

16a9099

chore: apply suggestions from code review

0354887

Co-authored-by: Alan Shaw <[email protected]>

fix: support optional proof and use did:web

a8b24c5

vasco-santos force-pushed the feat/trigger-content-claims-from-piece-cid branch from a3f4d8c to a8b24c5 Compare September 20, 2023 09:59

seed-deploy bot temporarily deployed to pr230 September 20, 2023 09:59 Inactive

vasco-santos mentioned this pull request Sep 20, 2023

Add Integration test for content claims from piece #234

Open

vasco-santos merged commit b68cb43 into main Sep 20, 2023
1 check passed

vasco-santos deleted the feat/trigger-content-claims-from-piece-cid branch September 20, 2023 10:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: trigger content claims from piece cid #230

feat: trigger content claims from piece cid #230

vasco-santos commented Sep 18, 2023 •

edited

Loading

vasco-santos Sep 19, 2023

vasco-santos Sep 19, 2023

seed-deploy bot commented Sep 19, 2023

alanshaw Sep 19, 2023

alanshaw Sep 19, 2023

vasco-santos Sep 19, 2023

alanshaw left a comment

alanshaw Sep 20, 2023

vasco-santos commented Sep 20, 2023

		Generated by [@ucanto/principal `EdSigner`](https://github.com/web3-storage/ucanto) via [`ucan-key`](https://www.npmjs.com/package/ucan-key)

		_Example:_ `MgCZG7EvaA...1pX9as=`

feat: trigger content claims from piece cid #230

feat: trigger content claims from piece cid #230

Conversation

vasco-santos commented Sep 18, 2023 • edited Loading

vasco-santos Sep 19, 2023

Choose a reason for hiding this comment

vasco-santos Sep 19, 2023

Choose a reason for hiding this comment

seed-deploy bot commented Sep 19, 2023

alanshaw Sep 19, 2023

Choose a reason for hiding this comment

alanshaw Sep 19, 2023

Choose a reason for hiding this comment

vasco-santos Sep 19, 2023

Choose a reason for hiding this comment

alanshaw left a comment

Choose a reason for hiding this comment

alanshaw Sep 20, 2023

Choose a reason for hiding this comment

vasco-santos commented Sep 20, 2023

vasco-santos commented Sep 18, 2023 •

edited

Loading